The Validation of Peer Review through Research Impact Measures and the Implications for Funding Strategies
نویسندگان
چکیده
There is a paucity of data in the literature concerning the validation of the grant application peer review process, which is used to help direct billions of dollars in research funds. Ultimately, this validation will hinge upon empirical data relating the output of funded projects to the predictions implicit in the overall scientific merit scores from the peer review of submitted applications. In an effort to address this need, the American Institute of Biological Sciences (AIBS) conducted a retrospective analysis of peer review data of 2,063 applications submitted to a particular research program and the bibliometric output of the resultant 227 funded projects over an 8-year period. Peer review scores associated with applications were found to be moderately correlated with the total time-adjusted citation output of funded projects, although a high degree of variability existed in the data. Analysis over time revealed that as average annual scores of all applications (both funded and unfunded) submitted to this program improved with time, the average annual citation output per application increased. Citation impact did not correlate with the amount of funds awarded per application or with the total annual programmatic budget. However, the number of funded applications per year was found to correlate well with total annual citation impact, suggesting that improving funding success rates by reducing the size of awards may be an efficient strategy to optimize the scientific impact of research program portfolios. This strategy must be weighed against the need for a balanced research portfolio and the inherent high costs of some areas of research. The relationship observed between peer review scores and bibliometric output lays the groundwork for establishing a model system for future prospective testing of the validity of peer review formats and procedures.
منابع مشابه
The Effect of Using Online Metacognitive Strategies Practice on EFL Learners’ Vocabulary Achievement: A Blended Approach
This study investigated a new blended approach for enhancing vocabulary achievement. To this end, the project used a convenience sample of 50 intermediate EFL learners ranging from 19-35 years of age from three intact classes studying English translation at university. They were randomly assigned to two groups of 25 students each. In the experimental group, the treatment consisted of providing ...
متن کاملA Review on the Editorial Peer Review
Background and Objectives: The editorial peer review has an important role in the publication of scientific articles. Peers or reviewers are those scholars who have the expertise regarding the topic of a given article. They critically appraise the articles without having any monetary incentives or conflicts of interest. The aim of this study was to determine the most important aspects of the ed...
متن کاملDesigning the synergistic leadership model for universities of Iran based on grounded theory
A review of the existing leadership literature in organizations shows that there is no comprehensive theoretical framework for synergistic leadership that is generally agreed upon. Therefore, this study seeks to identify and explain the dimensions of synergistic leadership in the form of a systematic model for Iranian universities using the grounded theory. Semi-structured interviews were used ...
متن کاملDesigning the synergistic leadership model for universities of Iran based on grounded theory
A review of the existing leadership literature in organizations shows that there is no comprehensive theoretical framework for synergistic leadership that is generally agreed upon. Therefore, this study seeks to identify and explain the dimensions of synergistic leadership in the form of a systematic model for Iranian universities using the grounded theory. Semi-structured interviews were used ...
متن کاملThe analysis of hazard identification and risk assessment studies with the approach to assessing risk control measures since 2001 to 2017: A systemic review
Abstract background and aims: Nowadays the growing complexity of technology and industry has led to vast changes over the last few decades. These changes, in addition to their positive and valuable effects, have also caused industrial accidents affecting human life and the environment. According to the ILO 2011 report, there are 340 million annual workplace accidents and 160 million occupation...
متن کامل